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FOREWORD 


This  assessment  of  the  implementation  of  IT  Capital  Planning  processes  has  been 
developed  on  behalf  of  the  Best  Practices  Subcommittee  of  the  Federal  Chief 
Information  Officer  (CIO)  Council’s  Capital  Planning  and  Information  Technology  (IT) 
Investment  Committee.  The  Subcommittee  is  pleased  to  present  the  results  of  an 
assessment  of  IT  capital  planning  activities  and  procedures. 

The  information  presented  herein  was  gathered  through  a  directed  questionnaire  and 
follow-up  interviews  conducted  with  ten  participating  agency  CIO’s  and  their  staffs 
between  April  22  and  May  1, 1998. 

The  questionnaire  findings,  as  well  as  the  input  from  the  interview  process,  are 
presented  for  the  mutual  benefit  of  the  participating  parties  and  other  government 
agencies  or  departments  that  may  benefit  from  “best  practices”  in  IT  capital  planning 
methods. 


This  assessment  is  a  joint  effort  of  the  Best  Practices  Subcommittee  and  LEADS 
Corporation.  The  Best  Practices  Subcommittee  wishes  to  thank  the  participating 
agencies  and  their  respective  CIO  staffs  for  completing  the  questionnaire  and  the 
scheduled  interviews  in  a  concise  and  timely  manner.  We  are  especially  grateful  to: 


Alan  Balutis 
David  Cristy 
Gary  Crowl 
Mark  Day 
Gary  Galloway 
Lee  Holcomb 
Brian  King 

Anne  F.  Thomson  Reed 
Dr.  Shereen  G.  Remez 
Kim  Taylor 


Department  of  Commerce 

Department  of  Housing  and  Urban  Development 

Department  of  Energy 

Environmental  Protection  Agency 

Department  of  State 

National  Aeronautics  and  Space  Administration 
U.S.  Coast  Guard 
U.S.  Department  of  Agriculture 
General  Services  Administration 
Department  of  Transportation 
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EXECUTIVE  SUMMARY 


Objectives  of  Current  Assessment 

This  assessment  of  IT  capital  planning  practices  is  a  high-level  look  at  how  well  the 
participating  agencies  are  progressing  in  the  implementation  of  the  first  and  best 
practices  identified  in  workshops  conducted  by  the  CIO  Council  in  1997.  The  emphasis 
in  this  assessment  is  directed  toward  ascertaining  progress  on  implementation  of  the 
“first”  and  “best  practices”  by  the  agencies  participating  in  the  pilot  plans.  The 
questionnaire  also  reflects  input  from  the  GSA’s  “IT  Capital  Planning  Guide”,  dated 
January  8,  1998,  and  the  OMB’s  “Capital  Planning  Guide”,  published  in  July  1997. 

Methodology 

The  assessment  was  conducted  through  a  directed  questionnaire  and  follow-up 
interviews  conducted  with  ten  participating  agency  CIO’s  or  their  staffs  between  April  22 
and  May  1,  1998. 

Participants 

Twelve  agencies  were  invited  to  participate,  however,  due  to  extremely  tight  time 
constraints,  only  ten  were  able  to  participate.  Participating  agencies  included: 

♦  Department  of  Agriculture  (USDA) 

♦  Department  of  Commerce  (DOC) 

♦  Department  of  Energy  (DOE) 

♦  Department  of  Housing  and  Urban  Development  (HUD) 

♦  Department  of  State  (DOS) 

♦  Department  of  Transportation  (DOT) 

♦  Environmental  Protection  Agency  (EPA) 

♦  General  Services  Administration  (GSA) 

♦  National  Aeronautics  and  Space  Administration  (NASA) 

♦  US  Coast  Guard  (USCG) 

Refer  to  Appendix  A  for  the  names  of  agency  participants 
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Method 

In  order  to  ensure  the  highest  possible  participation,  Dr.  Shereen  Remez,  GSA  CIO, 
and  chair  of  the  Best  Practices  Subcommittee,  issued  a  letter  to  potential  participants 
asking  for  their  cooperation  and  explaining  the  purpose  and  process  to  be  undertaken 
(Appendix  B). 

The  next  step  was  development  of  the  questionnaire.  The  GAO  Assessment  Guide 
provided  more  than  100  multiple  part  questions  about  the  various  phases  of  the  IT 
capital  planning  process,  and  served  as  an  initial  starting  point.  However,  in  order  to 
comply  with  extremely  tight  time  constraints,  it  was  necessary  to  cull  the  questions  to 
less  than  30.  The  selected  questions  were  then  mapped  to  the  “first  practices”  and 
initial  “best  practices”.  Appendix  C  presents  the  mapping  of  the  questions  to  the  “first 
practices”  and  initial  “best  practices”  and  the  blank  questionnaire. 

Questionnaires  were  distributed  to  participants  ahead  of  scheduled  interviews  so  that 
they  could  prepare  their  responses.  Ten  1-hour  interviews  were  conducted  in  eight 
working  days.  Participant’s  answers  were  then  compiled  and  interview  notes  analyzed 
in  order  to  produce  this  report. 


Summary  of  Findings 

Included  in  this  section  are  some  overall  observations  deduced  from  the  analysis  and 

specific  findings  related  to  the  first  practices  and  initial  best  practices. 

There  were  four  recurrent  themes  that  evolved  from  the  “best  practices”  assessment: 

♦  Quality  participation  or  “buy-in”  by  the  highest-level  agency  managers  in  the  IT 
capital  planning  process  is  necessary  for  successful  agency  implementation. 

♦  Collaboration  of  senior  management  (CEO(Director)/CFO/CIO)  is  necessary  to 
provide  focus  for  the  process. 

♦  Decision  making  at  the  lowest  appropriate  level  helps  drives  the  process. 

♦  Most  progress  has  been  made  in  the  development  of  IT  Investment  Selection 
processes. 
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BACKGROUND 


The  mission  of  the  Subcommittee,  as  defined  in  the  Federal  CIO  Strategic  Plan,  is: 

Continue  to  establish  guidance  for  “best  practices”  for  IT  Capital  Investment. 

The  measurement  of  this  objective  is  identifying  and  publishing  annually  relevant 
government  and  private  sector  best  practices.  Activities  are  defined  as: 

Completing  an  annual  process  of  documenting  “best  practices”  from 
government  and  the  private  sector,  updating  and  publishing  “Best  Practices", 
and  assessing  the  effectiveness  of  the  best  practices. 

The  process  began  with  the  Committee’s  “first  practices”  planning  session  on  February 
3  and  4, 1997.  This  planning  session  produced  the  Information  Technology  Investment: 
“First  Practices”,  booklet  which  highlighted  nine  outstanding  first  practices,  and  eleven 
success  factors  that  emerged  as  the  agencies  began  to  assess  what  works  in 
establishing  capital  planning  for  information  technology.  This  assessment  represents 
the  third  stage,  and  initial  assessment,  of  IT  capital  planning  “best  practices”. 

The  following  “first  practices”  emerged  from  the  February,  1997  CIO  Council  workshop: 

♦  Secure  senior  management  commitment  and  participation 

♦  Establish  an  executive-level  investment  review  board 

♦  Select  the  right  investments 

♦  Determine  costs  of  present  systems 

♦  Address  costs,  benefits,  and  risks  of  planned  investments 

♦  Provide  staff  analysis  to  the  investment  review  board  that  informs  decision  making 

♦  Make  decisions  as  needed 

♦  Control  initiatives  throughout  the  life  cycle 

♦  Evaluate  results  for  lessons  learned 


Implementing  Information  Technology  Capital  Planning  and  Investment  Processes:  “An  Assessment' 

4 


Critical  Success  Factors  and  Assessment  Findings 


The  following  section  summarizes  the  current  status  of  the  IT  capital  planning  and 
investment  processes  for  the  participating  federal  agencies.  The  assessment’s  findings 
focus  on  the  participants’  implementation  of  critical  success  factors  and  the  identification 
of  new  practices. 
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Critical  Success  Factor:  Secure  Senior  Management 
Commitment  and  Involvement 


How  will  we  demonstrate  the  successful  implementation  of  this  factor  in  our  IT 
capital  planning  and  investment  review  processes?  Agencies  report  that  we 
must: 

♦  Recognize  the  need  for  top  management  involvement 

♦  Use  the  budget  process  as  a  driver 

♦  Involve  functional  level  IT  executives 

♦  Involve  top  management  in  the  IT  Strategic  Planning  Board 

♦  Develop  partnerships  between  the  CIO,  CFO  and  CEO 

♦  Form  a  Business  Planning  Council 

♦  Have  open  communication  among  agency’s  top  leadership 

♦  Consider  establishing  Working  Capital  Funds 

♦  Have  strong  leadership  at  the  top 


Have  we  implemented  these  strategies?  If  not,  how  are  we  progressing? 

This  assessment  documents  a  strong  indication  that  agencies  embrace,  and  have  taken 
action  to  facilitate,  the  participation  of  senior  management  in  IT  capital  investment 
processes.  Nearly  ninety  percent  of  respondents  have  established  executive  level  IT 
investment  boards.  Senior  management  is  involved  in  ongoing  reviews,  although  the 
degree  and  frequency  of  involvement,  as  well  as  level  of  management  involved  varies 
among  the  agencies.  Some  agencies  employ  a  tiered  review,  while  others  use  a 
decentralized  review  structure.  In  some  instances,  only  certain  select  projects,  such  as 
those  defined  as  major,  critical,  or  cross-cutting,  receive  regular  attention  at  the  top 
executive  level  (Secretary,  Administrator,  etc.).  Three  agencies  specifically  noted  that 
their  top  executives  provide  strong,  focused  leadership  and  are  actively  involved  in  IT 
issues. 

Some  agencies  are  still  forging  the  partnering  of  the  CIO,  CFO  and  CEO  in  the  IT 
capital  planning  investment  process.  Almost  one-half  of  the  questions  in  the 
assessment  questionnaire  are  linked  to  this  critical  success  factor.  Agencies  have  been 
extremely  successful  in  some  of  the  areas,  while  other  areas  as  still  in  progress  or  need 
further  refinement. 

“Senior  management  insists  that  they  thoroughly  understand  a  proposed 

project  before  they  will  approve  it.” 


Implementing  Information  Technology  Capital  Planning  and  Investment  Processes:  “An  Assessment ” 

6 


What  are  we  doing  most  successfully? 

There  are  several  areas  that  all  or  most  of  the  agencies  agree  upon  and  have 

accomplished.  All  or  most  agencies: 

♦  Include  staff  from  program,  IT,  and  financial  offices  in  the  IT  Capital  planning  review 
processes. 

♦  Have  implemented,  or  are  close  to  completing  implementation  of,  formal  systematic 
processes  to  handle  proposed  project  submission  requirements  through  decision 
making  for  funding. 

♦  Have  adopted  an  integrated  team  approach  in  the  composition  of  the  review  groups 
by  including  representatives  from  the  different  functional  areas  of  program 
(operations),  finance,  and  IT. 

♦  Have  senior  management  review  summaries  on  project  costs,  benefits,  and  risks  of 
their  IT  investments. 

What  is  next?  Where  do  we  go  from  here  in  order  to  achieve  stronger  successes 

in  implementing  IT  Capital  Planning  processes? 

♦  Only  three-quarters  of  agencies  have  determined  who  has  responsibility  and 
authority  for  making  final  IT  related  funding  decisions. 

♦  Less  than  one  half  of  the  agencies  have  completed  procedures  defining  how  senior 
management  will  monitor  projects  in  investment  control  meetings;  another  three 
agencies  are  in  the  process  of  creating  procedures. 
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Critical  Success  Factor:  Establish  an  Executive  Level 
Investment  Review  Board 


How  should  an  Executive  Level  Review  Board  be  Structured  and  Operate? 
Agencies  report  we  must: 

♦  Build  on  existing  structures 

♦  Consider  the  need  for  multiple  boards 

♦  Consider  bureaus  at  decentralized  agencies 

♦  Use  different  approaches  at  centralized  agencies 

♦  Make  use  of  an  active,  energized  investment  review  board 

Have  we  implemented  these  strategies?  If  not,  how  are  we  progressing? 

Practically  speaking,  all  agencies  participating  in  this  assessment  have  adopted 
Executive-level  Investment  Review  Boards,  with  one  agency  utilizing  an  agency-level 
board  to  review  the  top  twenty  projects.  Most  agencies  employ  tiered  review  structures. 
The  assessing  entities  at  the  various  tiers  or  operational  units,  which  generally  involve 
senior  management,  review  both  proposed  and  on-going  projects.  Depending  on  the 
agency  and  project,  summary  reports  may  pass  up  through  the  agency,  and  other 
reviewing  bodies  may  conduct  additional  independent  assessments.  Under  certain 
conditions,  the  project  itself  may  become  the  charge  of  the  agency  and  executive-level 
IT  review  board. 

Almost  all  agencies  report  that  senior  management  conducts  at  least  an  annual  review 
or  assessment  of  summary  information  on  each  project’s  costs,  benefits,  and  risks.  The 
remaining  agencies  intend  to  implement  a  review  process  in  the  future. 

Although  only  just  over  half  report  conducting  scoring  exercises  to  evaluate  the 
strengths  and  weaknesses  of  proposals,  comments  from  the  remaining  agencies 
indicate  they  do  evaluate  proposals  for  this  purpose.  Methods  range  from  use  of  a 
simplified  procedure  of  “red,  yellow  and  green  light”,  to  conducting  extensive  analysis  at 
the  operating  unit  level.  A  few  agencies  do  not  pool  all  projects  from  their  operating 
units  for  the  purpose  of  determining  funding,  but  one  notes  that  comparisons  are  made 
on  the  proposed  project’s  contribution  to  the  agency’s  strategic  goals.  This  implies  that 
some  exercise  is  conducted  at  the  unit  level  to  evaluate  the  strengths  and  weaknesses 
of  competing  projects.  Almost  all  agencies  report  a  formal  systematic  process  for 
determining  priorities  and  making  funding  decisions. 

“All  major  IT  projects  which  are  eventually  included  in  the  Agency’s  annual  IT 
budget,  must  go  through  the  process  of  Information  of  Technology  Council, 
Council  of  Controllers,  and  Business  Technology  Council  endorsement" 
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What  are  we  doing  most  successfully? 

There  are  several  areas  that  all  or  most  of  the  agencies  agree  upon  and  have 

accomplished.  All  or  most  agencies: 

♦  Have  adopted  Executive-level  Investment  Review  Boards 

♦  Use  tiered  review  structures 

♦  Review  both  proposed  and  on-going  projects  at  the  tier  level 

♦  Have  senior  management  review  summary  project  information  at  least  annually 

♦  Have  a  formal  and  systematic  process  for  setting  priorities  and  making  funding 
decisions 

What  is  next?  Where  do  we  go  from  here  in  order  to  achieve  stronger  successes 

in  implementing  IT  Capital  Planning  processes? 

♦  Less  than  half  of  the  agencies  have  procedures  in  place  to  define  how  senior 
management  will  monitor  approved  projects. 

♦  Only  half  of  the  agencies  require  the  process  of  analyzing  and  comparing  IT  projects 
throughout  their  agency. 


Implementing  Information  Technology  Capital  Planning  and  Investment  Processes:  “An  Assessment’ 

9 


Critical  Success  Factor:  Select  the  Right  Investments 
(Using  Established  Criteria,) 


How  will  we  demonstrate  the  successful  implementation  of  this  factor  in  our  IT 
capital  planning  processes?  Agencies  report  that  we  must: 

♦  Define  thresholds  for  investment  planning  and  control 

♦  Include  agency  infrastructure  in  investment  portfolio 

♦  Develop  a  method  for  selecting  investments 

♦  Evaluate  investments  for  support  of  goals 

♦  Use  a  scorecard 

♦  Develop  portfolio  management  approaches 

♦  Standardize  reporting  formats 

♦  Develop  criteria  for  applying  decision  criteria 

Have  we  implemented  these  strategies?  If  not,  how  are  we  progressing? 

While  the  agencies  are  actively  building  the  tools  and  methodologies  that  will  ensure 
selection  of  the  best  investments  for  their  organizations,  all  the  necessary  elements  are 
not  yet  fully  in  place.  Agencies  appear  to  be  focusing  first  on  applying  investment 
evaluation  processes  to  new  project  proposals. 

Just  over  half  report  that  their  management  review  group  conducts  scoring  exercises  to 
evaluate  the  relative  strengths  and  weaknesses  of  proposals.  Of  the  remaining 
agencies,  one  replied  that  they  employ  a  simplified  procedure,  but  did  not  elaborate  on 
the  details  or  tools  used.  At  least  two  agencies  do  not  pool  all  projects  from  their 
operating  units  for  the  purpose  of  funding  determination,  although  one  of  these  noted 
that  comparisons  are  made  on  the  proposed  project’s  contribution  to  the  agency’s 
strategic  goals. 

“The  [Agency]  has  installed  the  l-TIPS  tracking  system  and  expect  it  will  be  a 
key  tool  in  developing  a  comprehensive  select,  control,  and  evaluate  process 
.  .  .  I-TIPS  is  expected  to  provide  a  great  deal  more  valid  information  and  the 
CIO’s  office  is  of  the  opinion  that  [they]  will  benefit  from  using  this  tool.  ” 


Just  under  half  of  the  respondents  report  they  maintain  and  track  data  on  their  current 
IT  spending  portfolio  by  category  of  investment  (operations  and  maintenance, 
infrastructure,  applications  and  systems  development). 
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Very  few  agencies  report  having  established  thresholds  for  cost/benefit  ratios,  ROI 
calculations,  and  risk  assessments  for  IT  project  screening,  although  some  are  in  the 
process  of  doing  so.  One  agency  noted  that  due  to  the  diversity  of  functions  among  its 
various  operating  units,  establishing  a  single  valid  set  of  agency-wide  thresholds  would 
be  difficult,  if  not  impossible.  This  agency  requires  ROI,  CBA,  risk  assessments,  and 
the  demonstration  of  clear  linkages  to  mission  goals  when  considering  investments. 
Several  agencies  have  concerns  regarding  the  suitability  of  a  single  standard  in 
organizations  with  diverse  operations. 

What  are  we  doing  most  successfully? 

♦  Only  one  agency  reports  exceptions  to  the  screening  criteria;  the  explanations  for 
the  exceptions  are  documented  and  forwarded  with  the  project  proposals. 

What  is  next?  Where  do  we  go  from  here  in  order  to  achieve  stronger  successes 
in  implementing  IT  Capital  Planning  processes? 

♦  Very  few  agencies  report  having  established  thresholds  for  cost/benefit  ratios,  ROI 
calculations,  and  risk  assessments  for  IT  project  screening. 

♦  Only  a  few  agencies  have  conducted  reviews  of  their  current  IT  spending  portfolios 
to  assess  alignment  with  mission  needs,  priorities,  strategic  direction,  or  major 
process  reengineering. 
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Critical  Success  Factor:  Determine  Cost  of  Present  Systems 


How  will  we  demonstrate  the  successful  implementation  of  this  factor  in  our  IT 
capital  planning  and  review  processes?  Agencies  report  that  we  must: 

♦  Determine  the  cost  of  current  operational  systems. 

♦  Address  the  full  life-cycle  cost  of  proposed  investments 

♦  Combine  benefits  of  existing  reporting 

♦  Establish  a  consistent  basis  for  cost  determination 

♦  Review  existing  projects 

♦  Establish  baseline  of  current  IT  assets 

Have  we  implemented  these  strategies?  If  not,  how  are  we  progressing? 

Currently,  just  over  half  indicate  that  they  have  been  able  to  gather  enough  data  to 
make  informed  decisions  on  continued  funding  of  an  existing  asset.  However,  this  is  a 
significant  improvement  from  the  original  “first  practices”  workshop  in  February  1997 
when  no  agency  reported  being  able  to  gather  this  information  satisfactorily. 

Almost  all  agencies  have  linked  the  capital  planning  process  with  the  budget  process, 
and  the  others  are  “in  the  process”  of  developing  the  link. 

Most  agencies  require  data  on  costs,  cost-benefit,  and  risk  analyses.  For  most,  senior 
management  reviews  or  assesses  summary  information  on  each  project's  costs, 
benefits  and  risks,  and  the  rest  intend  to  implement  this  practice  in  the  future. 

Only  one  agency  was  able  to  say  that  all  proposals  submitted  for  consideration  in  FY 
1999  contained  all  the  required  data,  although  several  indicated  that  50-80%  of  new 
proposals  submitted  included  all  required  data.  One  agency  noted  that  because  of 
timing,  not  all  requirements  had  been  established  at  the  time  of  the  FY  1999  budget  call. 
Another  agency  notes  deadlines  in  the  budget  cycle  (up  to  24  months  prior  to 
implementation)  necessitated  the  inclusion  of  some  projects  for  which  not  all  information 
had  been  gathered. 

“The  CIO’s  office  has  developed  a  database  of  IT  investments.  It  is  continually 
updated  vis-a-vis  the  Operation  Administrations’  budget  submissions,  the  5- 
Year  IT  plan,  and  periodic  status  reviews,  etc.  It  is  envisioned  that  the 
database  will  serve  as  the  basis  fora  Departmental  IT  portfolio.” 
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What  are  we  doing  most  successfully? 

♦  Processes  now  in  use  by  some  agencies  are  beginning  to  produce  sufficient  data  to 
evaluate  existing  projects. 

♦  The  capital  planning  process  and  the  budget  process  are  linked  in  most  agencies. 

♦  Senior  management  takes  an  active  role  in  reviewing  the  costs,  benefits  and  risks  of 
projects. 

♦  There  was  only  one  report  of  exempting  a  project  from  elements  of  the  screening 
criteria,  and  justification  for  the  exception  was  documented  and  forwarded  with  the 
project  proposal. 

What  is  next?  Where  do  we  go  from  here  in  order  to  achieve  stronger  successes 

in  implementing  IT  Capital  Planning  processes? 

♦  Only  a  few  agencies  report  having  completed  reviews  of  their  current  IT  spending 
portfolios. 

♦  Only  one  agency  was  able  to  say  that  all  proposals  submitted  for  consideration  in  FY 
1999  contained  all  the  required  data. 


New  practices  which  show  promise  for  the  future: 

♦  One  agency  has  developed  a  database  of  their  IT  systems  linked  to  their  budget  and 
strategic  plan.  Although  this  tool  currently  functions  as  an  inventory  of  all  IT  assets, 
they  intend  to  use  it  as  a  true  portfolio  of  their  IT  investments  in  the  very  near  future. 
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Critical  Success  Factor:  Address  Costs,  Benefits,  and  Risks 

of  Planned  Investments 


How  will  we  implement  these  measurements  in  our  IT  capital  planning  and 
investment  review  processes?  Agencies  report  that  we  must: 

♦  Include  all  relevant  costs 

♦  Choose  an  organizational  model  for  review 

♦  Predict  benefits  of  investments  that  accrue  in  the  near  term  rather  than  in  3-5  years 

♦  Analyze  multiple  investment  risk  categories 

Have  we  implemented  these  strategies?  If  not,  how  are  we  progressing? 

The  majority  of  the  agencies  advise  that  senior  management  reviews  summary 
information  on  each  project’s  costs,  benefits,  and  risks,  and  those  agencies  not 
currently  supplying  this  information  intend  to  do  so  in  the  future. 

Most  agencies  report  that  they  specifically  require  data  on  costs,  cost-benefit,  and  risk 
analyses  on  project  proposals,  particularly  major  projects  or  those  involving  Agency 
level  review.  A  number  of  respondents  note  that  data  requirements  vary  depending 
upon  the  size  of  the  IT  project.  Within  one  agency,  proposals  with  costs  of  $50  million 
or  greater  must  follow  the  data  requirements  of  Exhibit  300B  of  the  OMB’s  Circular  A- 
11,  while  smaller  projects  are  subject  to  a  subset  of  those  data  elements.  Oral 
comments  by  several  of  the  interviewees  indicate  that  analysis  requirements  are 
relaxed  for  certain  types  of  projects,  such  as  those  which  are  mandated,  or  where  it  is 
not  possible  to  calculate  a  quantitative  measurement  of  some  criterion,  such  as 
benefits. 

“Both  [pre-existing  and  new]  Select,  Control,  and  Evaluate  models  require 
performance  of  complete  and  extensive  cost-benefit  analysis,  including  net 
present  value  computation  and  risk  analysis  for  larger  projects.” 

Among  the  decentralized  agencies,  and  those  with  tiered  review  structures,  data 
requirements  are  frequently  established  by  the  various  departments  or  operating  units. 
Thus,  it  is  unclear  as  to  how  extensively  these  measurements  are  addressed  for 
projects  reviewed  and  approved  at  the  sub-agency  or  department  level. 

Only  one  agency  advised  that  all  IT  proposals  submitted  for  funding  consideration  in  FY 
1999  included  all  of  the  requisite  data  (including  analyses  on  costs-benefits  and  risks). 
On  average,  slightly  more  than  half  of  the  new  projects  submitted  for  consideration  were 
fully  compliant  with  all  data  requirements  in  another  four  agencies.  One  agency  with 
published  guidelines  reported  that  the  FY  1999  budget  cycle  began  prior  to  agency-wide 
adoption  of  the  IT  capital  planning  process,  therefore,  not  all  proposals  submitted  for 
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funding  consideration  contained  all  requisite  data.  Another  responded  that  the  question 
assumed  portfolio  management  at  the  agency  level,  a  function  that  is  performed  only  for 
certain  investments  at  this  agency,  all  others  being  managed  at  the  program  level. 

What  are  we  doing  most  successfully? 

♦  Virtually  all  of  the  agencies  have  completed  implementation  of  a  formal,  systematic 
process  for  determining  priorities  and  making  funding  decisions. 

♦  Agencies  are  adhering  to  their  selection  processes.  Only  one  respondent  said  that 
they  had  made  an  exception  to  their  screening  criteria  for  new  projects,  and  noted 
that  the  rational  for  the  exception  was  fully  documented. 

What  is  next?  Where  do  we  go  from  here  in  order  to  achieve  stronger  successes 
in  implementing  IT  Capital  Planning  processes? 

♦  Only  one  agency  advised  that  all  IT  proposals  submitted  for  funding  consideration  in 
FY  1999  included  all  of  the  requisite  data. 
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Critical  Success  Factor:  Provide  Staff  Analysis  that  Informs 

Decision-Making 


How  will  we  ensure  that  sufficient  relevant  data  is  available  for  staff  to  provide 
analyses  that  informs  decision-making?  Agencies  recognize  that  we  should: 

♦  Make  the  business  case 

♦  Address  cost,  risk,  and  benefits 

♦  Be  brief 

Have  we  implemented  these  strategies?  If  not,  how  are  we  progressing? 

The  agencies  appear  to  be  in  the  process  of  accumulating  data,  including  historical  data 
on  existing  projects,  which  staff  can  utilize  to  build  evaluations  for  use  by  their 
investment  boards.  Clearly,  agencies  recognize  the  value  of  collecting  data  for  analysis 
and  questionnaire  responses  provide  insight  on  their  progress  in  making  the  business 
case.  Most  agencies  have  completed,  or  are  in  the  process  of,  identifying  the  benefits 
of  each  IT  investment  using  quantitative  and  or  qualitative  data  relating  directly  to 
mission  support  and  performance  improvement.  The  majority  requires  that  data  on 
costs,  benefits,  and  risk  analyses  are  included  in  documentation  supporting  major 
proposed  projects.  However,  not  all  projects  are  subject  to  the  same  analysis  criterion. 

“ The  Planning  Staff  is  employed  full-time  reviewing  [proposed  project]  plans. 

They  will  also  farm-out  project  plans  for  additional  review  if  they  deem  a 
specialized  [project]  warrants  insight  from  a  specialist.” 

As  noted  previously,  only  one  agency  was  able  to  fulfill  the  data  requirements  for  all 
projects  submitted  for  funding  consideration  in  FY  1999.  Four  agencies  reported  that 
half,  or  better,  of  the  new  proposals  submitted  met  their  agency’s  data  documentation 
requirements. 

The  omission  of  some  supporting  analysis  may  be  partially  explained  by  timing  issues. 
Several  participants  noted  that  in  some  instances  good  business  judgement  dictates 
inclusion  of  a  project  in  the  budget  even  though  the  agency  is  still  gathering  information 
to  complete  their  evaluations.  Otherwise,  the  advanced  timing  of  the  budget  cycle  (up 
to  24  months  prior  to  implementation)  would  exclude  or  detrimentally  delay  some 
projects  that  may  be  pivotal  to  a  mission  objective  and  will  prove  valid  upon  subsequent 
evaluation. 

The  consistent  collection  of  all  relevant  data  is  critical  to  providing  staff  analysis  that 
ensures  informed  decision  making.  The  majority  of  respondents  advise  that  senior 
management  review  summary  information  on  a  proposed  project’s  costs,  benefits,  and 
risks.  However,  all  desirable  data  for  weighing  the  advantages  of  funding  a  new 
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acquisition  against  continuing  the  funding  of  an  existing  asset,  may  not  yet  be  compiled 
and  readily  available  for  analysis.  Only  a  third  of  the  reporting  agencies  have 
completed,  or  substantially  completed,  reviews  of  their  current  IT  spending  portfolio  to 
assess  alignment  with  mission  needs  priorities,  strategic  direction,  or  major  process 
reengineering. 

Interview  comments  indicate  that  educating  personnel  at  various  levels  on  both  the 
benefits  of,  and  disciplines  involved  in,  IT  capital  planning  is  critical  to  future  success. 
Sufficient  time  has  not  elapsed  for  agencies  to  define  and  disseminate  data 
requirements,  and  train  personnel  in  the  techniques  of  quantitative  analysis,  that  would 
yield  all  the  data  necessary  to  facilitate  meaningful  analysis  for  this  year’s  budget  cycle. 
However,  several  agencies  indicated  that  within  the  next  year  they  anticipate  significant 
progress  as  a  result  of  the  planning  and  effort  expended  in  the  last  year. 

What  are  we  doing  most  successfully? 

♦  Agencies  are  adhering  to  their  screening  criteria  for  proposed  projects:  to  date,  only 
one  agency  has  encountered  an  exception  to  their  criteria. 

♦  Senior  management  reviews  summary  information  on  each  project’s  costs,  benefits, 
and  risks  in  the  majority  of  the  agencies. 

What  is  next?  Where  do  we  go  from  here  in  order  to  achieve  stronger  successes 
in  implementing  IT  Capital  Planning  processes? 

♦  Only  a  third  of  the  reporting  agencies  have  completed,  or  substantially  completed, 
reviews  of  their  current  IT  spending  portfolio. 
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Critical  Success  Factor:  Make  Decisions  When  Needed 


How  will  we  show  successful  implementation  of  this  factor  in  our  ITCP  and 
Investment  Review  processes?  Agencies  report  that  we  must: 

♦  Be  prepared  to  stop  a  project  when  necessary 

♦  Be  prepared,  also,  to  help  in  making  changes 

♦  Establish  corporate  decision  making  infrastructure 

Have  we  implemented  these  strategies? 

Most  agencies  appear  to  recognize  the  importance  of  providing  relevant  data  to  senior 
management  that  fosters  prudent  and  timely  decisions  regarding  continuance  of  a 
project.  Most  agencies  report  that  they  notify  senior  management  of  gaps  between 
estimates  and  actual  performance,  and  that  they  document  the  reasons  for  variances. 
Only  one  agency  reported  any  instance  of  an  exception  to  their  screening  criteria,  and 
the  justification  for  the  exception  was  documented  and  submitted  with  the  proposal. 
This  appears  to  imply  that  the  agencies  are  working  to  ensure  that  all  adequate  data 
(including  omission  disclosure)  are  collected  for  informed  decision  making. 

“The  Program  Management  Council  employs  a  rigorous  set  of  program 
management  procedures.  They  set  thresholds  for  large  projects,  measure, 
and  evaluate  carefully  and  will  cancel  a  project  that  experiences  unjustified 
cost  over-runs.” 


What  are  we  doing  most  successfully? 

♦  Most  agencies  notify  senior  management  of  gaps  between  estimated  and  actual 
performance  and  then  document  the  reasons  for  variances. 

♦  Most  agencies  are  briefing  upper  level  management  concerning  the  results  of  the 
preliminary  selection  process. 

What  needs  improvement?  Where  do  we  go  from  here? 

♦  Only  slightly  over  half  of  the  agencies  regularly  update  project  business  cases  to 
reflect  current  costs,  risks,  and  interim  performance  results  for  use  in  investment 
control  meetings. 
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New  practices  which  show  promise  for  the  future: 

♦  One  agency  uses  project  business  case  information  which  is  updated  annually  in  the 
selection  briefing  process,  and  which  will  be  done  monthly  in  the  fully  implemented 
control  and  evaluation  process. 
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Critical  Success  Factor:  Control  Initiatives  Throughout  The 

Life  Cycle 


How  will  we  demonstrate  the  successful  implementation  of  this  factor  in  our  ITCP 
and  Investment  Review  processes?  Agencies  report  that  we  must: 

♦  Use  the  portfolio  to  manage  costs  and  oversee  implementation 

♦  Control  investments,  independent  of  the  selection  process 

♦  Ensure  that  problems  are  surfaced  and  discussed  when  they  occur 

♦  Monitor  results  or  outcomes 

♦  Manage  high  risk  projects 

♦  Develop  preview  of  milestones/review  schedule 

♦  Remain  faithful  to  the  scheduled  project  reviews 

Have  these  strategies  been  implemented?  If  not,  how  are  we  progressing? 

The  structure  of  senior  management  involvement  in  ongoing  reviews  varies.  Some 
agencies  employ  a  tiered  review,  while  others  use  a  decentralized  review  structure.  In 
some  instances,  only  certain  selected  projects,  such  as  those  defined  as  major,  critical, 
or  cross-cutting,  receive  regular  attention  at  the  top  executive  level  (Secretary, 
Administrator,  etc.).  However,  all  agencies  include  staff  from  program,  IT,  and  financial 
offices  in  the  IT  capital  planning  review  process.  Some  agencies  employ  an  integrated 
project  team  approach  throughout  the  project  life  cycle. 

“We  currently  have  high  level  milestones  and  cost  information  for  each  project 
and  are  using  A-11  requirements  and  project  information  in  greater  detail  so 
that  projects  not  meeting  cost  or  schedule  goals  are  more  quickly  identified.” 

What  is  being  done  most  successfully?  Most  agencies  report  that: 

♦  Senior  management  is  apprised  of  deviations  between  estimated  and  actual 
performance. 

What  needs  Improvement?  Where  do  we  go  from  here  to  achieve  stronger 
successes  in  implementing  ITCP  processes? 

♦  Only  slightly  over  half  of  the  agencies  say  they  are  able  to  obtain  sufficient  data  on 
actual  versus  projected  costs  to  support  informed  decision  making  regarding  project 
continuation. 

♦  Barely  half  of  the  agencies  regularly  update  project  business  cases  to  reflect  current 
costs,  risks,  and  interim  performance  results,  for  use  in  investment  control  meetings. 
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New,  or  newly  used,  practices  which  show  promise  for  the  future: 

♦  Two  agencies  mentioned  the  use  of  earned  value  project  management  techniques  to 
control  time  and  cost  schedules  for  their  projects. 
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Critical  Success  Factor:  Evaluate  Results  for  Lessons 

Learned 


How  are  we  showing  the  successful  implementation  of  this  factor  in  our  ITCP  and 
Investment  Review  processes?  Agencies  report  that  we  must 

♦  Tie  proposed  investments  to  program  initiatives 

♦  Develop  an  explicit  relationship  to  the  GPRA  Performance  Plan  and  report 

♦  When  measuring  performance,  keep  your  eye  on  the  prize 

♦  Critique  Select  and  Control  phases  during  evaluation 

♦  Incorporate  lessons  learned  into  the  process 

♦  Incorporate  evaluation  results  into  overall  IT  business  practices 

♦  Agree  up-front  on  what  is  to  be  evaluated 

♦  Select  the  right  staff  to  perform  evaluations 

Have  we  implemented  these  strategies?  If  not,  how  are  progressing? 

Assessment  of  overall  responses  indicates  that  the  creation  of  procedures  for 
evaluation  and  feedback,  including  the  collection  of  information  pertinent  to  their 
evaluations,  will  require  additional  time.  In  some  instances,  agencies  have  not  yet  had 
the  opportunity  to  track  a  project  through  the  selection  and  control  phases,  much  less 
evaluate  the  results  of  either  the  project  or  the  process. 

“Our  Pilot-Guide  was  revised  after  nine  months  and  re-issued  September  ’97. 

Based  on  lessons  learned— -we  expect  more  [revisions]  in  the  future.” 


What  are  we  doing  most  successfully?  Most  agencies  report: 

♦  Some  degree  of  implementation  in  the  publishing  of  guidelines  defining  where  data 
on  IT  projects  will  be  maintained. 

♦  Identifying  benefits  of  each  IT  investment  using  quantitative/qualitative  information  to 
ensure  that  benefits  relate  directly  to  mission  support  and  performance 
improvement. 

What  is  next?  What  is  needed  to  achieve  stronger  successes  in  implementing 

ITCP  processes? 

♦  Only  slightly  more  than  half  of  the  agencies  use  performance  measures  and 
cost/benefit  analysis  to  substantiate  support  for  the  business  mission. 
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NEW  PRACTICES 


The  preceding  sections  provide  insight  on  the  status  of  implementation  of  IT  capital 
planning  processes,  with  a  particular  emphasis  on  the  spread  of  “first”  and  “best 
practices”  previously  identifed.  But  what  about  the  future?  What  new  approaches  have 
been  identified?  The  following  new  practices,  which  were  not  identified  in  prior  forums, 
are  being  used  by  some  agencies  in  their  IT  capital  planning  processes: 

♦  One  agency  uses  project  business  case  information  which  is  updated  annually  in  the 
selection  briefing  process. 

♦  Two  agencies  mentioned  the  use  of  earned  value  project  management  techniques  to 
control  time  and  cost  schedules  for  their  projects. 

♦  One  agency  has  developed  a  database  of  their  IT  systems  linked  to  their  budget  and 
strategic  plan.  Although  this  tool  currently  functions  as  an  inventory  of  all  IT  assets, 
they  intend  to  use  it  as  a  true  portfolio  of  their  IT  investments  in  the  very  near  future. 


NEXT  STEPS 

This  assessment  is  an  early  step  in  a  continuing  evaluation  process.  It  is  a  picture  of 
where  implementation  for  IT  capital  planning  currently  stands  for  nine  agencies.  The 
next  step  requires  a  more  in-depth  assessment  and  the  involvement  of  all  federal 
agencies  that  need  to  meet  Clinger-Cohen  IT  capital  planning  requirements.  ’’Best 
practices’”  will  only  provide  their  greatest  benefits  when  all  potential  participants  are 
involved  and  sharing  in  the  information  process. 
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APPENDIX  A  -  SURVEY  PARTICIPANTS 


AGENCY  PARTICIPANTS 

Department  of  Agriculture  (USDA) 

Anne  F.  Thomson  Reed,  Chief  Information 
Officer,  Office  of  the  CIO 

David  Allardyce,  Office  of  the  CIO 

Program,  Planning  and  Management 

Division 

Department  of  Commerce  (DOC) 

Alan  Balutis,  Director  for  Budget, 
Management  and  Information  and  Chief 
Information  Officer 

Department  of  Energy  (DOE) 

Gary  Crowl,  Director  of  HR42 

Office  of  Policy,  Planning,  and  Mission 
Analysis 

Department  of  Housing  and  Urban 
Development  (HUD) 

David  Cristy,  Director  of  Information 

Resource  Management  Policy  and 
Management  Division 

Department  of  State  (DOS) 

Gary  Galloway,  Senior  Policy  Analyst 

Officer  of  the  Chief  Information  Officer 

Department  of  Transportation 

Diane  Litman,  Manager,  Information 
Resources  Management 

Kim  Taylor,  Director  of  Information 

Resources  Management 

Environmental  Protection  Agency  (EPA) 

Mark  Day,  Deputy  Chief  Information  Officer 

General  Services  Administration 

Dr.  Shereen  G.  Remez,  Chief  Information 
Officer,  Office  of  the  CIO 

L.  Diane  Savoy,  Acting  Assistant  Chief 
Information  Officer,  Office  of  the  CIO 

Reginald  Hardman,  Acting  Director,  Center 
for  IT  Capital  Planning 

Michael  Kernich,  Center  for  IT  Capital 
Planning 

William  McVay,  Center  for  IT  Capital 

Planning 

National  Aeronautics  and  Space 
Administration  (NASA) 

Lee  Holcomb,  Chief  Information  Officer 

Eva  Layne 

US  Coast  Guard  (USCG) 

Brian  King,  Chief  Office  of  Architecture  and 
Planning 

John  Theimer,  IT  Strategic  Planner,  Office 
of  Architecture  and  Planning,  Systems 
Planning  Branch 
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APPENDIX  B  -  LETTER  TO  PARTICIPANTS 


[Addressee] 

Chief  Information  Officer 


Dear 


In  January  1998,  the  Chief  Information  Officer  (CIO)  Council  established  a  Strategic  Plan  which  outlined 
the  goals,  objectives,  and  activities  for  the  committees  off  the  Council.  The  Capital  Planning  and  IT 
Investment  Committee  has  an  objective  to  “continue  to  establish  guidance  for  best  practices  for  IT  capital 
investments”.  The  Best  Practices  Subcommittee  was  established  to  lead  this  effort. 

In  follow-up  to  the  IT  Capital  Planning  Pilot  Program,  which  established  and  published  both  first  practices 
and  best  practices,  the  Subcommittee  will  assess  the  effectiveness  of  the  best  practices  implemented  at 
both  the  pilot  agencies  and  other  agencies  which  have  developed  and  implemented  a  process.  This 
effort  is  scheduled  to  be  completed  by  May  1998.  V\fe  are  pursuing  an  aggressive  schedule  to  meet  our 
objectives  outlined  in  the  CIO  Strategic  Plan. 

As  we  discussed  and  decided  during  the  Best  Practices  Subcommittee  meeting  on  March  23,  1998,  we 
have  employed  a  contractor,  Leads  Corporation,  to  assist  us  with  the  assessment.  They  have  been 
provided  the  Information  Technology  Investment:  First  Practices ,  the  Implementing  Best  Practices, 
Strategies  at  Work,  the  GAO  Assessing  Risks  and  Returns:  A  Guide  for  Evaluating  Federal  Agencies  IT 
Investment  Decision-Making,  and  the  OMB  Capital  Programming  Guide  to  uses  as  background,  reference 
and  guiding  material.  The  contractor,  in  partnership  with  the  Subcommittee,  is  developing  a 
questionnaire  to  gather  information.  To  quickly  complete  this  questionnaire,  they  will  be  contacting  you  to 
schedule  a  one-hour  interview  with  you  and/or  any  staff  to  obtain  information  regarding  the  Capital 
Planning  and  IT  Investment  program  in  your  agency.  You  will  be  provided  an  advance  copy  of  the 
questionnaire  at  least  five  days  prior  to  your  interview.  You  may  also  provide  the  contractor  with  any 
documentation  or  information  you  consider  relevant.  The  information  from  the  interviews  and  any 
documentation  will  be  used  by  the  contractor  to  develop  and  provide  an  Assessment  Report.  The 
contractor  will  brief  the  Best  Practices  Subcommittee  on  the  findings  and  analysis.  We  will  then  publish 
and  present  this  document  to  the  Capital  Planning  and  IT  Investment  Committee  and  to  the  CIO  Council. 

Any  assistance  you  can  provide  to  ensuring  that  the  designated  interviewee  completes  the  questionnaire 
prior  to  the  interview  and  is  available  for  the  interview  will  greatly  enhance  the  process  and  ensure  as 
many  agencies  as  possible  are  represented  in  the  assessment.  I  want  to  thank  you  for  your  participation 
and  cooperation. 

If  you  have  any  questions,  please  contact  Ms.  L.  Diane  Savoy,  Acting  Assistant  Chief  Information  Officer 
of  Planning  and  IT  Architecture  on  202-501-3535. 

Sincerely, 


Shereen  G.  Remez 
Chief  Information  Officer 
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APPENDIX  C  -  QUESTIONNAIRE 


Appendix  C  presents  the  mapping  of  the  27  questions  in  the  questionnaire  to  the  nine 
first  practices  and  the  related  initial  best  practices.  A  sample  copy  of  the  questionnaire 
follows. 
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APPENDIX  C 


Question/Practice  Cross-Tabulation  Matrix 


APPENDIX  C  -  QUESTIONNAIRE 


Federal  CIO  Council  Best  Practices  Assessment 

Questionnaire 

for  Information  Technology  Capital  Planning 

Per  the  letter  of  April  8,  1998  from  Dr.  Shereen  G.  Remez,  GSA  CIO,  LEADS 
Corporation  is  forwarding  the  following  questionnaire  to  your  attention  and  has 
previously  scheduled  interviews  with  you  and  your  staffs  to  clarify  any  questions  you 
may  have  regarding  your  completion  of  the  questionnaire.  If  possible,  we  would 
appreciate  receiving  your  completed  questionnaires  via  E-mail  or  fax  prior  to  the 
scheduled  interview;  E-mail  (iohn.roberts@leadscorD.com).  fax  (703)  769-5660, 
attention  John  Roberts. 

The  questionnaire  was  developed  for  the  Best  Practices  Subcommittee  of  the  CIO 
Council's  Capital  Planning  and  IT  Investment  Committee.  The  questionnaire  utilizes 
questions,  which  are  directly  related  to  ‘Best  Practices’  cited  by  participating  agencies  in 
the  “Implementing  Best  Practices  Strategies  at  Work”,  workshop  conducted  in  July 
1997.  The  questions  also  coincide  with  the  ‘First  Practices’  procedures  highlighted  in 
the  “Information  Technology  Investment:  First  Practices”  workshop  which  initiated  the 
best  practices  implementation  in  February  1997. 


Thank  you  for  your  participation.  Your  responses  are  important  as  they  will  be  used  to 
establish  a  “cross-agency”  inventory  of  Best  Practices  which  can  be  a  powerful  tool  in 
the  development  of  your  IT  capital  planning  and  budgeting.  The  information  you  provide 
wil[  be  tepj  strictly  confidential. _ 


A  list  of  the  First  Practices  is  included  for  your  reference  on  the  last  page  of  this 
questionnaire. 


Basis:  First  Practice  one  and  Best  Practices: 

1 .  Where  is  the  agency  in  the  creation  (or  revision),  of  defining  a  formal  process  for 
the  submission  and  screening  of  new  funding  proposals? 

□  Complete  through  documentation  and  training  of  personnel. 

□  In  process,  _ %  completed 


Comments: 
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Basis:  First  Practice  three  and  Best  Practices: 


2.  Has  the  Agency  defined  thresholds  for  cost/benefit  ratios,  ROI  calculations,  risk 
assessments,  etc.  for  IT  project  screening?  Please  provide  a  brief  description  of 
your  process  for  addressing  these  areas. 

Yes  □  No  □  □  In  process, _ %  completed 

Description  of  process: 

Basis:  First  Practices  one  and  two,  and  Best  Practices: 

3.  Does  the  Agency  have  a  formal  systematic  process  for  determining  priorities  and 
making  funding  decisions? 

Yes  □  No  □  □  In  process, _ %  completed 

If ‘Yes’,  please  provide  a  brief  description: 

Basis:  First  Practices  one  and  Best  Practices: 

4.  Has  the  Agency  determined  who  has  the  responsibility  and  authority  for  making 
final  IT-related  funding  decisions? 

Yes  □  No  □  □  In  process, _ %  completed 

Comments: 

Basis  for  questions  5  and  6:  Overall  First  and  Best  Practices: 

5.  Has  the  Agency  conducted  a  review  (in-house,  or  via  outside  consultant)  of  its 
current  IT  spending  portfolio  to  assess  alignment  with  mission  needs,  priorities, 
strategic  direction,  or  major  process  reengineering? 

Yes  □  No  □  □  In  process, _ %  completed 

Comments: 
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6.  Has  the  Agency  published  guidelines  defining  where  data  on  IT  projects  will  be 
maintained? 

Yes  □  No  □  □  In  process, _ %  completed 


Comments: 


Basis:  First  Practices  four,  five,  six,  and  Best  Practices 

7.  What  are  your  Agency’s  requisite  data  requirements  for  project  proposals?  (Please 
list) 


For  IT  proposals  submitted  for  funding  consideration  in  FY  1999,  was  all  requisite 
data  costs/benefits/risks)  submitted  in  accordance  with  prescribed  Agency 
requirements? 

YesQ  NoQ  □  Not  applicable 


If  ‘No’,  approximate  percentage  of  proposed  projects  that  were  submitted  with  all 
requisite  support  data? 

Comments: 


Basis:  First  Practices  three,  six  and  nine,  and  Best  Practices 

8.  Have  benefits  of  each  IT  investment  been  identified  using  quantitative  and/or 

qualitative  data/information  that  relate  directly  to  mission  support  and  performance 
improvement? 

Yes  □  No  □  □  In  process, _ %  completed 

Comments: 


Basis:  First  Practice  three  and  Best  Practices 

9.  Does  the  agency  maintain  and  track  data  on  its  current  IT  spending  portfolio  by 
category  of  investment  such  as  operations  and  maintenance,  infrastructure, 
applications  and  systems  development,  hardware  acquisitions,  etc.? 

Yes  □  No  □  □  Under  review 
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Comments: 

Basis:  Overall  First  and  Best  Practices 

10.  If  exceptions  are  being  made  to  screening  criteria,  is  the  explanation  documented 
and  forwarded  with  the  project  proposal? 

Yes  □  No  □  □  Not  applicable;  no  exceptions  encountered 

Comments: 

Basis:  First  Practice  two  and  Best  Practices 

11.  Is  the  process  for  analyzing  and  comparing  IT  projects  required  throughout  the 
Agency? 

Yes  □  No  □ 

If  ‘No’,  will  the  process  be  implemented  agency-wide  in  the  future? 

Yes  □  No  □ 

Comments: 

Basis  for  questions  12  and  13:  First  Practices  one  and  eight  and  Best  Practices 

12.  Who  is  involved  in  ongoing  project  reviews  and  decisions? 

13.  Do  the  review  groups  include  staff  from  program,  IT,  and  financial  offices? 

Yes  □  No  □ 

If  No,  will  other  offices  be  included  in  the  future? 

Yes  □  No  □ 

Basis:  First  Practices  four  and  eight  and  Best  Practices 

14.  Has  the  agency  been  able  to  obtain  sufficient  data  on  projected  versus  actual 
costs,  and  interim  results,  to  facilitate  informed  decisions  about  project 
continuation? 
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YesQ 


No  □ 


Comments: 

Basis:  First  Practice  one  and  two,  and  Best  Practice 

15.  Do  existing  procedures  define  how  approved  projects  will  be  monitored  by  senior 
management  in  regular  investment  control  meetings? 

Yes  □  No  □  □  In  process, _ %  completed 

Comments: 

Basis:  Best  Practices 

16.  Do  procedures  exist  for  aggregating  data/information  across  all  major  IT  projects 
(or  spending  categories)  in  order  to  compile  an  overall  organization  track  record  on 
costs  and  benefits  attributable  to  IT? 

Yes  □  No  □ 

If ‘No’,  is  process  being  considered? 

Yes  □  No  □ 


Comments: 


Basis:  First  Practices  one,  seven,  eight,  nine,  and  Best  Practices 

17.  Is  senior  management  notified  of  gaps  or  differences  between  estimates  and 
actuals,  and  are  explanatory  factors  documented  for  positive  or  negative 
variances? 

Yes  □  No  □ 

Comments: 


Basis:  First  Practices  seven,  eight,  nine,  and  Best  Practices 

1 8.  Is  the  information  in  project  business  cases  updated  to  reflect  the  current  state 
(including  costs  to  date,  current  risks  and  mitigation  plans,  interim  benefit  or 
performance  results  achieved,  etc.)  for  use  in  investment  control  meetings? 
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YesQ 


No  □ 


Comments: 

Basis  for  questions  19  and  20:  First  Practices  nine  and  Best  Practices 

19.  Does  the  organization  have  a  defined,  documented  process  for  conducting  post¬ 
implementation  reviews  (PIR)  of  IT  projects? 

Yes  □  No  □  □  In  process, _ %  completed 

Comments: 

20.  If  the  answer  to  question  #19  is  ‘Yes’,  is  the  purpose  of  the  PIR  process  clearly 
explained  and  communicated? 

Yes  □  No  □ 

Does  the  process  specify  when  PIR's  are  to  be  conducted? 

Yes  □  No  □ 

Basis:  First  Practices  one,  two,  four,  five,  six,  and  Best  Practices 

21 .  Does  Senior  Management  review/assess  summary  information  on  each  project’s 
costs,  benefits,  and  risks? 

Yes  □  No  □ 

If  ‘Yes’,  how  frequently? 

If  ‘No’,  does  the  Agency  intend  to  implement  this  practice? 

Yes  □  No  □ 

Who  in  Senior  Management  participates  (Be  specific): 

Basis:  First  Practices  one,  two,  three,  and  Best  Practices 

22.  Does  the  management  review  group  conduct  scoring  exercises  to  evaluate  the 
relative  strengths  and  weaknesses  of  proposals? 

Yes  □  No  □ 
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If  ‘No’,  How  are  the  relative  strengths  graded? 


Comments: 

Basis  for  questions  23  and  24:  First  Practices  nine  and  Best  Practices 

23.  Does  the  organization  have  a  process  for  evaluating  current  decision-making 
processes  and  suggesting  changes  to  these  processes  based  on  lessons  that  are 
learned  from  investment  control  reviews? 

Yes  □  No  □  □  In  process, _ %  completed 

Comments: 

24.  Is  there  a  process  for  refining  or  updating  the  selection  criteria  (both  screening  and 
ranking)  based  on  lessons  that  are  learned  from  investment  control  reviews? 

Yes  □  No  □  □  In  process, _ %  completed 

Comments: 

Basis:  First  practices  four  and  Best  Practices 

25.  Are  your  capital  planning  and  Agency  budget  process  linked?  Please  explain 
how. 

Basis:  First  Practices  one  and  Best  Practices 

26.  Do  you  have  an  executive  level  IT  Investment  Review  Board? 

Yes  □  No  □ 

Comments: 

27.  Do  you  have  a  Business  Planning  Council  or  similar  body? 

Yes  □  No  □ 

If  ‘Yes’,  which  members  of  senior  management  participate? 
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FIRST  PRACTICES 


1 .  Secure  senior  management  commitment  and  participation. 

2.  Establish  an  executive-level  investment  review  board. 

3.  Select  the  right  investments  (using  established  criteria.) 

4.  Determine  costs  of  present  systems. 

5.  Address  costs,  benefits,  and  risks  of  planned  investments. 

6.  Provide  staff  analysis  to  the  investment  review  board  that  informs  decision-making. 

7.  Make  decisions  when  needed. 

8.  Control  initiatives  throughout  the  life  cycle. 

9.  Evaluate  results  for  lessons  learned. 
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APPENDIX  D  -  RESPONSE  PROFILE 


Federal  CIO  Council  Information  Technology  Capital  Planning 
Best  Practices  Assessment  Questionnaire 

Basis:  First  Practice  one  and  Best  Practices: 

1.  Where  is  the  agency  in  the  creation  (or  revision),  of  defining  a  formal  process  for 
the  submission  and  screening  of  new  funding  proposals? 

QUANTITY  RESPONSE 

5  Yes,  complete  through  documentation  and  training  of  personnel. 

1  In  process,  90%  completed 

1  In  process,  80%  completed 

1  In  process,  25%  completed 

1  No 


Basis:  First  Practice  three  and  Best  Practices: 

2.  Has  the  Agency  defined  thresholds  for  cost/benefit  ratios,  ROI  calculations,  risk 
assessments,  etc.  for  IT  project  screening?  Please  provide  a  brief  description  of 
your  process  for  addressing  these  areas. 


QUANTITY  RESPONSE 

3  Yes 

2  In  process,  50%  completed 

4  No 


Basis:  First  Practices  one  and  two,  and  Best  Practices: 

3.  Does  the  Agency  have  a  formal  systematic  process  for  determining  priorities  and 
making  funding  decisions? 

QUANTITY  RESPONSE 

6  Yes 

2  100% 

1  In  process,  90%  completed 
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Basis:  First  Practices  one  and  Best  Practices: 

4.  Has  the  Agency  determined  who  has  the  responsibility  and  authority  for  making 
final  IT-related  funding  decisions? 


QUANTITY 

RESPONSE 

5 

Yes 

1 

100% 

1 

In  process,  50%  completed 

1 

No 

1 

Unclear,  assumed  ‘no’ 

Basis  for  questions 

5  and  6:  Overall  First  and  Best  Practices. 

5.  Has  the  Agency  conducted  a  review  (in-house,  or  via  outside  consultant)  of  its 

current  IT  spending  portfolio  to  assess  alignment  with  mission  needs,  priorities, 
strategic  direction,  or  major  process  reengineering? 

QUANTITY 

RESPONSE 

2 

Yes 

1 

In  process,  90%  completed 

5 

No 

1 

Unclear,  assumed  ‘no’ 

6.  Has  the  Agency  published  guidelines  defining  where  data  on  IT  projects  will  be 

maintained? 

QUANTITY 

RESPONSE 

5 

Yes 

1 

In  process,  95%  completed 

1 

In  process,  50%  completed 

2 

No 

Basis:  First  Practices  four,  five,  six,  and  Best  Practices 

7.  What  are  your  Agency’s  requisite  data  requirements  for  project  proposals?  (Please 
list) 

RESPONSE 

♦  Agency  1 

♦  A  description  of  the  Agency  requirements  based  on  missions,  including 
linkage  to  the  Strategic  Plan,  the  Program’s  strategic  Plans  and  the 
operating  unit’s  strategic  plans. 

♦  A  prioritized  list  of  requirements,  including  time  phasing. 
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(Continuation  of  responses  to  Question  7) 

♦  A  description  of  existing  and  in-process  Agency  capabilities,  including 
condition  and  inputs  from  appropriate  ongoing  functional  assessment 
activities. 

♦  A  description  of  related  Federal,  academic,  industry,  and  other  capabilities, 
including  potential  opportunities  for  partnering. 

♦  An  evaluation  of  requirements  and  capabilities,  including  partnering  and 
funding  profiles. 

♦  An  investment  strategy  with  proposed  roles  for  the  various  elements  of  the 
organization. 

♦  Cost,  schedule,  and  performance  metrics  for  the  entire  life  cycle. 

♦  Documentation  of  how  the  investment  meets  “Raines’  Rules” 

♦  Criteria  for  the  Agency  level  decision  process  include:  Proposed  program 
content  and  objectives 

♦  Agency  2 

♦  Level  of  data  detail  varies  by  the  dollar  value  of  the  project.  Generally,  the 
data  requirements  for  proposals  with  costs  of  $50M  &  over  follow  OMB 
Circular  A-1 1,  Exhibit  300B  data  elements.  A  subset  of  the  data  elements 
in  Exhibit  300B  is  required  for  projects  with  costs  below  $50M. 

♦  Aaencv  3 

♦  Business  case  developed  collaboratively;  significant  requirements  analysis 

♦  Agency  4 

♦  All  project  proposals  must  specifically  address  each  of  the  'Raines'  Rules  in 
addition  to  information  requested  in  strategic  and  operational  IT  Planning 
calls.  Major  IT  projects  require  specific  evaluation  criteria  that  assess  return 
and  risk.  Return  factors  include  mission  effectiveness,  customer  needs, 
ROI,  expected  improvement.  Risk  factors  include  investment  size,  project 
longevity,  management  risk,  and  technical  risk. 

♦  Agency  5 

♦  Extensive  list  of  documentation,  which  depending  upon  size  of  project 
requires:  business  case,  feasibility  study,  functional  requirements, 
alternative  analysis,  ROI,  cost/benefit  analysis,  acquisition  plan,  project 
plan,  risk  assessment  and  mitigation  plan,  security  plan,  pilot/prototype 
plans,  technical  documentation. 

♦  Agency  6 
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♦  Under  development 


(Continuation  of  responses  to  Question  7) 

♦  Agency  7 

♦  Can  vary  from  year  to  year  because  of  changes  in  business  priorities  and 
processes.  SDM  contributes  documentation  standards.  New  1-Tips  process 
will  contribute  rigor. 


For  IT  proposals  submitted  for  funding  consideration  in  FY  1999,  was  all  requisite 
data  costs/benefits/risks)  submitted  in  accordance  with  prescribed  Agency 
requirements? 


QUANTITY  RESPONSE 

1  Yes 

6  No 

2  Not  Applicable 


If  ‘No’,  approximate  percentage  of  proposed  projects  that  were  submitted  with  all 
requisite  support  data? 


QUANTITY 

1 

1 

1 

1 

1 


RESPONSE 

80% 

75-80% 

50% 

None 

Other 


Basis:  First  Practices  three,  six  and  nine,  and  Best  Practices 

8.  Have  benefits  of  each  IT  investment  been  identified  using  quantitative  and/or 
qualitative  data/information  that  relate  directly  to  mission  support  and  performance 
improvement? 


QUANTITY 

5 

1 

1 

1 

1 


RESPONSE 

Yes 

Most,  80% 

75% 

In  progress,  50% 
Unclear,  assumed  ‘no’ 
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Basis:  First  Practice  three  and  Best  Practices 

9.  Does  the  agency  maintain  and  track  data  on  its  current  IT  spending  portfolio  by 
category  of  investment  such  as  operations  and  maintenance,  infrastructure, 
applications  and  systems  development,  hardware  acquisitions,  etc.? 


QUANTITY 

4 

1 

3 

1 


RESPONSE 

Yes 

Under  review 
No 

No  reply 


Basis:  Overall  First  and  Best  Practices 

10.  If  exceptions  are  being  made  to  screening  criteria,  is  the  explanation  documented 
and  forwarded  with  the  project  proposal? 

QUANTITY  RESPONSE 

1  Yes 

1  No  reply 

7  Not  applicable 


Basis:  First  Practice  two  and  Best  Practices 

11.  Is  the  process  for  analyzing  and  comparing  IT  projects  required  throughout  the 
Agency? 

QUANTITY  RESPONSE 

5  Yes 

4  No 


If  ‘No’,  will  the  process  be  implemented  agency-wide  in  the  future? 

QUANTITY  RESPONSE 

3  Yes 


Basis  for  questions  12  and  13:  First  Practices  one  and  eight  and  Best  Practices 

12.  Who  is  involved  in  ongoing  project  reviews  and  decisions? 

RESPONSE 


♦  Deputy  Secretary,  Senior  Department  Program  Managers,  Site  IT  Managers 

♦  IRM  Peer  Group  and  IRM  Board 
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(Continuation  of  responses  to  Question  12) 

♦  CIO,  DEP.  CIO,  CFO/ASA,  Dir.  Of  Policy  and  Strategic  planning,  CIOs  from  the 
four  largest  operating  units,  (at  the  Dept,  level) 

♦  TIBEC,  TIBWG,  CFO,  CIO,  IT  staff,  project  manager 

♦  CIO,  CFO,  Comptroller,  Dep.  CIO,  Sen.  SES  Leadership,  Sen.  IT  staff 

♦  Dep.  Sec.,  Asst.  Secretaries,  CIO,  Deputies  of  Operating  Administrations, 
cognizant  program  officials 

♦  Agency  level  IT,  program  and  financial  people,  Dept.  Under  and  Asst. 
Secretaries,  Budget  &  Finance,  Office  of  the  CIO. 

♦  Review  teams  including  organization  staff  and  the  OCIO.  ITC  reviews  some 
projects.  All  projects  reviewed  by  ITC/COC,  and  BTC. 

♦  Program  Management  Council  for  program  specific  investments.  Major  agency¬ 
wide  infrastructure  projects  approved  through  CIC  and  PMC. 


13.  Do  the  review  groups  include  staff  from  program,  IT,  and  financial  offices? 

QUANTITY  RESPONSE 

9  Yes 


If  ‘No’,  will  other  offices  be  included  in  the  future? 
N/A 


Basis:  First  Practices  four  and  eight  and  Best  Practices 

14.  Has  the  agency  been  able  to  obtain  sufficient  data  on  projected  versus  actual 
costs,  and  interim  results,  to  facilitate  informed  decisions  about  project 
continuation? 


QUANTITY 

5 

3 

1 


RESPONSE 

Yes 

No 

Unknown 
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Basis:  First  Practice  one  and  two,  and  Best  Practice 

15.  Do  existing  procedures  define  how  approved  projects  will  be  monitored  by  senior 
management  in  regular  investment  control  meetings? 


QUANTITY 

4 

1 

1 

1 

2 


RESPONSE 

Yes 

In  process,  70%completed 
In  process,  50%completed 
In  process,  33%  completed 
No 


Basis:  Best  Practices 

16.  Do  procedures  exist  for  aggregating  data/information  across  all  major  IT  projects 
(or  spending  categories)  in  order  to  compile  an  overall  organization  track  record  on 
costs  and  benefits  attributable  to  IT? 

QUANTITY  RESPONSE 

4  Yes 

5  No 


If  ‘No’,  is  process  being  considered? 

QUANTITY  RESPONSE 

3  Yes 

2  No 


Basis:  First  Practices  one,  seven,  eight,  nine,  and  Best  Practices 

17.  Is  senior  management  notified  of  gaps  or  differences  between  estimates  and 
actuals,  and  are  explanatory  factors  documented  for  positive  or  negative 
variances? 


QUANTITY  RESPONSE 

7  Yes 

2  No 
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Basis:  First  Practices  seven,  eight,  nine,  and  Best  Practices 

18.  Is  the  information  in  project  business  cases  updated  to  reflect  the  current  state 
(including  costs  to  date,  current  risks  and  mitigation  plans,  interim  benefit  or 
performance  results  achieved,  etc.)  for  use  in  investment  control  meetings? 

QUANTITY  RESPONSE 

5  Yes 

4  No 


Basis  for  questions  19  and  20:  First  Practices  nine  and  Best  Practices 

19.  Does  the  organization  have  a  defined,  documented  process  for  conducting  post¬ 
implementation  reviews  (PIR)  of  IT  projects? 

QUANTITY  RESPONSE 

5  Yes 

4  No 


20.  If  the  answer  to  question  #19  is  ‘Yes’,  is  the  purpose  of  the  PIR  process  clearly 
explained  and  communicated? 

QUANTITY  RESPONSE 

5  Yes 


Does  the  process  specify  when  PIR's  are  to  be  conducted? 

QUANTITY  RESPONSE 

4  Yes 

1  No 


Basis:  First  Practices  one,  two,  four,  five,  six,  and  Best  Practices 

21.  Does  Senior  Management  review/assess  summary  information  on  each  project’s 
costs,  benefits,  and  risks? 

QUANTITY  RESPONSE 

7  Yes 

2  No 

If ‘Yes’,  how  frequently? 

QUANTITY  RESPONSE 

4  Yearly 

3  Vary 

1  No  reply 
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(Continuation  of  Question  21) 

If  ‘No’,  does  the  Agency  intend  to  implement  this  practice? 

QUANTITY  RESPONSE 

2  Yes 


Who  in  Senior  Management  participates  (Be  specific): 

RESPONSE 


♦  ECIM  consists  of  Deputy  Secretary,  CIO,  CFO,  and  Assistant  Secretary  of 
Programs.  Assessments  with  detailed  performance  measurements  are 
submitted  to  Office  of  CIO  on  pre-agreed  schedule.  Office  of  CIO  reports  to 
ECIM  trend  analysis  of  all  IT  initiatives  within  agency. 

♦  Administrator,  members  of  Business  Technology  Council  (BTC),  Information 
Technology  Council  (ITC),  and  Council  of  Controllers.  The  ITC  includes  CIOs 
of  all  organizations,  a  representative  of  the  Inspector  General,  and  three 
regional  representatives.  The  COC  includes  the  controllers  of  the 
organizations.  The  BTC  includes  all  Commissioners  of  Service  &  staff  offices, 
three  regional  administrators,  Associate  Administrator  of  Public  Affairs,  Agency 
Chief  of  Staff,  Agency  CIO,  Deputy  CIO,  Agency  CFO,  Deputy  Administrator 
and  the  Administrator. 

♦  At  the  Department  level,  projects  meeting  certain  criteria  such  as  life  cycle 
costs  >25M,  political  sensitivity,  crosscutting  system,  etc.  are  subject  to  review 
by  the  CIO  &  CITRB.  At  the  department  level,  CITRB  include  CIO,  Deputy 
CIO,  CFO/ASA,  Director  of  Policy  &  Strategic  Planning,  CIOs  from  the  four 
largest  operating  units  plus  two  other  operating  unit  CIOs  on  a  rotating  basis, 
Director  for  Acquisition  Management  and  senior  executives  as  invited. 

♦  Budget,  Program,  and  IT  staff 

♦  CFO,  CIO,  Technical  Investment  Board  Working  Group  (TIBWG)  consisting 
primarily  of  program  managers,  Technical  Investment  Board  Executive  Council 
(TIBEC)  includes  Secretary,  Deputy  Secretary,  executive/principal  staff 

♦  Deputy  Secretary,  Assistant  Secretary,  CIO’s  office,  Deputy  of  operating 
Administrations,  cognizant  program  officials,  and  certain  Departmental  offices. 

♦  At  Agency  level,  Deputy  Administrator,  CIO,  senior  managers  from  the 
Strategic  Enterprises.  At  lower  levels,  Program  Management  Council. 
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Basis:  First  Practices  one,  two,  three,  and  Best  Practices 

22.  Does  the  management  review  group  conduct  scoring  exercises  to  evaluate  the 
relative  strengths  and  weaknesses  of  proposals? 

QUANTITY  RESPONSE 

6  Yes 

3  No 


If  ‘No’,  How  are  the  relative  strengths  graded? 

RESPONSE 


♦  IT  proposals  from  the  individual  programs  are  not  pooled  for  evaluation  or 
ranking  purposes.  Extensive  analyses  are  conducted  on  proposed  IT 
investments  within  the  individual  programs  or  projects  to  determine  their 
appropriateness  to  the  program’s  requirements. 

♦  A  simplified  procedure  of  rating  a  proposed  IT  investment  as  Green  (Go),  Red 
(stop),  or  Yellow  (caution,  more  information  required.)  is  utilized.  Again,  IT 
proposals  from  the  different  operating  units  do  not  compete  against  each  other 
for  funding. 

♦  Comparisons  are  made  based  on  the  proposed  project’s  contribution  to  the 
Agency’s  strategic  goals  and  corporate  strategies.  Generally,  IT  proposals 
from  the  different  operating  units  do  not  compete  against  each  other  for 
funding. 


Basis  for  questions  23  and  24:  First  Practices  nine  and  Best  Practices 

23.  Does  the  organization  have  a  process  for  evaluating  current  decision-making 
processes  and  suggesting  changes  to  these  processes  based  on  lessons  that  are 
learned  from  investment  control  reviews? 

QUANTITY  RESPONSE 

5  Yes 

1  In  process,  50%  completed 

3  No 


24.  Is  there  a  process  for  refining  or  updating  the  selection  criteria  (both  screening  and 
ranking)  based  on  lessons  that  are  learned  from  investment  control  reviews? 

QUANTITY  RESPONSE 

3  Yes 

1  In  process,  90%  completed 
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1 

4 


In  process 
No 


Basis:  First  practices  four  and  Best  Practices 

25.  Are  your  capital  planning  and  Agency  budget  process  linked?  Please  explain 
how. 


QUANTITY 

RESPONSE 

8 

Yes 

1 

In  process,  50%  completed 

Basis:  First  Practices  one  and  Best  Practices 

26. 

Do  you  have 

an  executive  level  IT  Investment  Review  Board? 

QUANTITY 

RESPONSE 

7 

Yes 

1 

No 

1 

Comments  only 

27. 

Do  you  have 

a  Business  Planning  Council  or  similar  body? 

QUANTITY 

RESPONSE 

5 

Yes 

1 

No 

2 

No  response 

1 

Question  unclear,  assumed  ‘no’ 

If  ‘Yes’,  which  members  of  senior  management  participate? 

RESPONSE 

♦  Administrator,  Deputy  Administrator,  Regional  Administrators,  Associate 
Administrators,  CFO,  CIO,  GC,  IG 

♦  All  Commissioners  of  Service  &  staff  offices,  three  regional  administrators, 
Associate  Administrator  of  Public  Affairs,  Agency  Chief  of  Staff,  Agency  CIO, 
Deputy  CIO,  Agency  CFO,  Deputy  Administrator  and  the  Administrator. 

♦  Senior  IT  official  from  each  operating  administration,  Agency  CIO,  other 
members  not  enumerated 
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APPENDIX  E  -  ACRONYM  LIST 


AIMD 

BPR 

BTC 

CBA 

CEO 

CFO 

CIB 

CIO 

CURB 

COC 

COTS 

DOC 

DOD 

DOE 

DOT 

DPG 

DRB 

EITRB 

EPA 

FY 

GAO 

GPRA 

GSA 

HUD 

l-TIPS 

IPT 

IRM 

IT 

ITC 

ITM 

NASA 

OMB 

PIR 

PMC 

R&D 

ROI 

USCG 

USDA 

WAN 

Y2K 


Accounting  and  Information  Management  Division  (GAO) 

Business  Process  Reengineering 

Business  Technology  Council 

Cost/Benefit  Analysis 

Chief  Executive  Officer 

Chief  Financial  Officer 

Corporate  Investment  Board 

Chief  Information  Officer 

Commerce  Information  Technology  Review  Board 

Council  of  Controllers 

Commercial  Off-the-Shelf 

Department  of  Commerce 

Department  of  Defense 

Department  of  Energy 

Department  of  Transportation 

Defense  Planning  Guidance 

Defense  Resources  Board 

Executive  Information  Technology  Investment  Review  Board 
Environmental  Protection  Agency 
Fiscal  Year 

General  Accounting  Office 

Government  Performance  and  Results  Act 

General  Services  Administration 

Department  of  Housing  and  Urban  Development 

Information  Technology  Investment  Portfolio  System 

Integrated  Project  Team 

Information  Resources  Management 

Information  Technology 

Information  Technology  Council 

Information  Technology  Management 

National  Aeronautics  and  Space  Administration 

Office  of  Management  and  Budget 

Post  Implementation  Review 

Program  Management  Council 

Research  and  Development 

Return  on  Investment 

U.S.  Coast  Guard 

Department  of  Agriculture 

Wide  Area  Network 

Year  2000 
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